Skip to content

Support Upstash and Vercel KV environment variables for Redis authentication#57

Merged
jgwill merged 11 commits intomainfrom
copilot/fix-redis-authentication-error
Nov 7, 2025
Merged

Support Upstash and Vercel KV environment variables for Redis authentication#57
jgwill merged 11 commits intomainfrom
copilot/fix-redis-authentication-error

Conversation

Copy link
Contributor

Copilot AI commented Nov 7, 2025

coaia tash and coaia fetch failed with authentication errors because the code didn't support standard environment variables from Vercel KV and only recognized Upstash direct variables.

Changes

Environment variable handling (coaiamodule.py)

  • Support standard Upstash direct variables: UPSTASH_REDIS_REST_URL and UPSTASH_REDIS_REST_TOKEN
  • Support Vercel KV REST API variables: KV_REST_API_URL and KV_REST_API_TOKEN
  • Support Vercel KV connection strings: KV_URL and REDIS_URL (format: rediss://default:password@host:6379)
  • Parse both HTTPS URL format and Redis connection string format
  • Load .env variables into os.environ while preserving OS environment priority
  • Removed non-standard UPSTASH_REST_API_* variables that caused confusion

Debugging capability (coaiamodule.py, coaiacli.py)

  • Add --verbose flag to tash and fetch commands
  • Display connection details: host, port, SSL, masked password (last 4 chars only)
  • Update error messages to mention all supported variable naming conventions

Configuration priority

  1. Upstash direct (UPSTASH_REDIS_REST_*)
  2. Vercel KV REST API (KV_REST_API_*)
  3. Vercel KV connection strings (KV_URL, REDIS_URL)
  4. Traditional Redis variables (REDIS_HOST, REDIS_PASSWORD)
  5. Config files (./coaia.json, ~/coaia.json)

Examples

Upstash Direct:

$ cat .env
UPSTASH_REDIS_REST_URL=https://my-redis.upstash.io
UPSTASH_REDIS_REST_TOKEN=my_token

$ coaia tash KEY "value" --verbose
Connecting to Redis server:
  Host: my-redis.upstash.io
  Port: 6379
  SSL: True
  Password: ***oken
  Status: Connection established successfully

Vercel KV (REST API):

$ cat .env
KV_REST_API_URL=https://my-redis.upstash.io
KV_REST_API_TOKEN=my_token

$ coaia tash KEY "value" --verbose

Vercel KV (Connection String):

$ cat .env
KV_URL=rediss://default:password@host.upstash.io:6379

$ coaia tash KEY "value" --verbose

Documentation

Added comprehensive documentation:

  • ENVIRONMENT_VARIABLES_REFERENCE.md - Quick reference for all supported variables
  • UPDATE_NOTES.md - Migration guide from non-standard variables
  • SUPPORTED_VARIABLES.txt - Visual ASCII reference
  • Updated REDIS_FIX_DOCUMENTATION.md with correct variable names

Backward compatible - existing configurations continue working unchanged.

Fixes #55

Original prompt

This section details on the original issue you should resolve

<issue_title>ISSUE_251107_coaia_tash_fetch.md</issue_title>
<issue_description># Issue: coaia tash and coaia fetch commands failing with Redis authentication error

Problem Description

The coaia tash and coaia fetch commands are currently failing to execute successfully, returning an "invalid username-password pair or user is disabled" error. This occurs even when no .env file is explicitly present in the current working directory, suggesting a potential misconfiguration or incorrect handling of Redis connection parameters.

Observed Behavior

When attempting to use coaia tash (e.g., coaia tash TMP -F README.md -T 33) or coaia fetch <key>, the following error message is displayed:

invalid username-password pair or user is disabled
Key: TMP  was just saved to memory.

Despite the message indicating the key was saved, subsequent coaia fetch commands for the same key also fail with the same authentication error. This indicates that the operation is not completing successfully due to a Redis authentication issue.

Expected Behavior

  1. The coaia tash and coaia fetch commands should successfully interact with the configured Redis instance.
  2. The commands should correctly prioritize environment variables for Redis connection, specifically UPSTASH_REDIS_REST_URL and UPSTASH_REDIS_REST_TOKEN.
  3. If a .env file exists in the current working directory (/src/coaiapy/), these variables should be read from it and used for Redis authentication.
  4. If no .env file is present, the commands should fall back to other configured methods (e.g., ~/.coaia.json or other global configurations) without authentication errors.

Reproduction Steps

  1. Ensure no .env file is present in /src/coaiapy/.
  2. Execute coaia tash TMP -F README.md -T 33.
  3. Observe the "invalid username-password pair or user is disabled" error.
  4. Execute coaia fetch TMP.
  5. Observe the same authentication error.

Relevant Environment Variables

The following environment variables are critical for Redis connection and should be correctly processed by the coaia tool:

  • UPSTASH_REDIS_REST_URL
  • UPSTASH_REDIS_REST_TOKEN

These variables should be looked for in the .env file within the current working directory (/src/coaiapy/) first.

Suggested Solution/Investigation Areas for Copilot Agents

Copilot agents should investigate the following:

  1. Environment Variable Loading: Verify how coaia loads and prioritizes environment variables, especially from .env files. Ensure that python-dotenv or a similar mechanism is correctly implemented and functioning.
  2. Redis Client Configuration: Examine the coaia codebase to understand how the Redis client is initialized and how the UPSTASH_REDIS_REST_URL and UPSTASH_REDIS_REST_TOKEN are passed to it. There might be an issue with how these credentials are being formatted or used during the connection attempt.
  3. Fallback Mechanism: Confirm that the fallback logic for Redis configuration (when .env variables are not present) is correctly implemented and does not lead to authentication failures.
  4. Error Handling: Improve error messaging to provide more specific details about why the authentication failed (e.g., which credential was invalid, which configuration source was used).
  5. Testing: Implement unit and integration tests to ensure robust handling of Redis connection parameters and environment variables under various scenarios (e.g., .env present, .env absent, invalid credentials).</issue_description>

<agent_instructions>It is not the first time I assign you that issue, you seems not to have fixed it.

maybe add a flag "--view-redis-url" that will show the value of the server or just a --verbose at least to 'coaia tash' and 'coaia fetch'

my .env has the environment values UPSTASH_REST_API_URL and UPSTASH_REST_API_TOKEN but the "coaia tash" dont store the values in that REDIS server but probably another one that is not desired, it should look in the ./.env for those values

I dont know what is hapenning but you have some more work todo, you closed PR #56 failed to fix that</agent_instructions>

Comments on the Issue (you are @copilot in this section)

@jgwill # Complementary Analysis: Adjustments for `coaia tash` and `coaia fetch` Redis Issue

Analysis of Likely Root Cause

The primary issue appears to stem from an incomplete or incorrect mechanism for loading and prioritizing Redis connection credentials, particularly the UPSTASH_REDIS_REST_URL and UPSTASH_REDIS_REST_TOKEN environment variables. The error message "invalid username-password pair or user is disabled" strongly suggests that the Redis client is either not receiving the correct credentials or is attempting to connect with default/empty credentials.

Given the user's expectation that .env files should...


✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

Copilot AI self-assigned this Nov 7, 2025
@jgwill
Copy link
Owner

jgwill commented Nov 7, 2025

After I added

UPSTASH_REST_API_URL="https://full-...-12634.upstash.io"

UPSTASH_REST_API_TOKEN="ATF.....MzQ"
UPSTASH_REDIS_REST_TOKEN="AT.....MzQ"

  • Something is confusing which default to look for in UPSTASH_REST_API_TOKEN, UPSTASH_REDIS_REST_TOKEN....
  • UPSTASH_REDIS_REST_TOKEN has a meaningless REDIS in it !!

I hate these confusion...

Copilot AI and others added 6 commits November 7, 2025 20:18
- Support both UPSTASH_REDIS_REST_* and UPSTASH_REST_API_* naming conventions
- Load .env file variables into os.environ while respecting OS env priority
- Add --verbose flag to 'coaia tash' and 'coaia fetch' commands
- Display detailed Redis connection info in verbose mode
- Update error messages with both environment variable naming conventions

Co-authored-by: jgwill <23141173+jgwill@users.noreply.github.com>
- Document both UPSTASH_REDIS_REST_* and UPSTASH_REST_API_* naming conventions
- Explain configuration priority order (OS env > .env > config files)
- Provide troubleshooting guide for common issues
- Include usage examples for different scenarios
- Describe verbose mode debugging capabilities

Co-authored-by: jgwill <23141173+jgwill@users.noreply.github.com>
- Handle short passwords safely in verbose output (avoid IndexError)
- Fix line continuation indentation for better readability
- Add consistent spacing after commas in function calls
- Improve password masking logic to handle edge cases

Co-authored-by: jgwill <23141173+jgwill@users.noreply.github.com>
- Document that password is masked before logging
- Add explanatory comments for CodeQL analysis
- Password masking is intentional security measure (only shows last 4 chars)

Co-authored-by: jgwill <23141173+jgwill@users.noreply.github.com>
- Document all changes made to resolve Redis authentication issues
- Include root cause analysis and solutions
- List all testing performed and security considerations
- Provide usage examples and migration guidance
- Confirm backward compatibility

Co-authored-by: jgwill <23141173+jgwill@users.noreply.github.com>
- Show concrete examples of error messages before and after fix
- Demonstrate verbose mode output with real examples
- Include troubleshooting scenarios with solutions
- Provide quick start guide for users
- Create comprehensive visual comparison document

Co-authored-by: jgwill <23141173+jgwill@users.noreply.github.com>
@jgwill
Copy link
Owner

jgwill commented Nov 7, 2025

@copilot because we use Vercel.com, the connection with UPSTASH as storage outputs these variables :

KV_URL="rediss://default:A...MzQ@full-...-12...4.upstash.io:6379"
KV_REST_API_URL="https://full-...-12...4.upstash.io"
KV_REST_API_TOKEN="A...MzQ"
KV_REST_API_READ_ONLY_TOKEN="A...ToQ"
REDIS_URL="rediss://default:AT...MzQ@full-...-12...4.upstash.io:6379"
  • When we directly use upstash.com (without passing thru vercel marketplace)
UPSTASH_REDIS_REST_URL="https://....upstash.io"
UPSTASH_REDIS_REST_TOKEN="A..."

It gives us what we need to support and whatever else exist that dont match these should be removed because it gets confusing what should be the environment variables. Remove anything else from .env and of course, we still falling back to the ./coaia.json or $HOME/coaia.json property: "jtaleconf": host,port,password,ssl

Copilot AI changed the title [WIP] Fix Redis authentication error in coaia tash and fetch commands Support both UPSTASH_REST_API_* and UPSTASH_REDIS_REST_* environment variables for Redis authentication Nov 7, 2025
Copilot AI requested a review from jgwill November 7, 2025 20:34
…confusing UPSTASH_REST_API_*

- Remove UPSTASH_REST_API_URL/TOKEN (non-standard, confusing)
- Add support for Vercel KV variables: KV_REST_API_URL/TOKEN, KV_URL, REDIS_URL
- Update priority order: UPSTASH_REDIS_REST_* > KV_* > REDIS_* > config
- Parse both REST API format (https://) and connection strings (rediss://)
- Update documentation to reflect correct supported variables
- Update error messages with clearer guidance

Co-authored-by: jgwill <23141173+jgwill@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Nov 7, 2025

@copilot because we use Vercel.com, the connection with UPSTASH as storage outputs these variables :

KV_URL="rediss://default:A...MzQ@full-...-12...4.upstash.io:6379"

...

Added full support for Vercel KV variables as you specified:

Supported formats:

  • KV_REST_API_URL + KV_REST_API_TOKEN
  • KV_URL (connection string: rediss://default:password@host:6379)
  • REDIS_URL (alternative connection string)
  • UPSTASH_REDIS_REST_URL + UPSTASH_REDIS_REST_TOKEN (Upstash direct)

Priority order maintained:

  1. Upstash direct variables
  2. Vercel KV variables
  3. Traditional REDIS_* variables
  4. ./coaia.json or ~/coaia.json (jtaleconf)

The code now parses both HTTPS URLs and rediss:// connection strings correctly. Non-standard UPSTASH_REST_API_* variables removed to eliminate confusion.

Fixed in commit 7e3f80f

After I added

UPSTASH_REST_API_URL="https://full-...-12634.upstash.io"
...

You're right - the UPSTASH_REST_API_* variables were confusing and non-standard. I've removed them.

The code now only supports standard variables:

  • Upstash: UPSTASH_REDIS_REST_URL / UPSTASH_REDIS_REST_TOKEN
  • Vercel: KV_REST_API_URL / KV_REST_API_TOKEN, KV_URL, REDIS_URL

Fixed in commit 7e3f80f

Copilot AI and others added 3 commits November 7, 2025 20:41
- Create ENVIRONMENT_VARIABLES_REFERENCE.md with quick reference for all supported variables
- Add UPDATE_NOTES.md explaining the correction from non-standard to standard variables
- Document priority order and when to use each format
- Include migration guide from old UPSTASH_REST_API_* to correct variables
- Add troubleshooting section for common issues

Co-authored-by: jgwill <23141173+jgwill@users.noreply.github.com>
- Document all changes made to address user feedback
- List removed non-standard variables and added standard ones
- Show platform compatibility matrix
- Include before/after comparison for users
- Provide clear next steps and verification methods

Co-authored-by: jgwill <23141173+jgwill@users.noreply.github.com>
- Create ASCII art reference showing all supported variables
- Clearly mark removed non-standard variables
- Show priority order with visual indicators
- Include testing examples and migration guide
- Easy to read format for quick reference

Co-authored-by: jgwill <23141173+jgwill@users.noreply.github.com>
Copilot AI changed the title Support both UPSTASH_REST_API_* and UPSTASH_REDIS_REST_* environment variables for Redis authentication Support Upstash and Vercel KV environment variables for Redis authentication Nov 7, 2025
@jgwill jgwill marked this pull request as ready for review November 7, 2025 20:51
@jgwill jgwill merged commit 2e90173 into main Nov 7, 2025
@jgwill jgwill deleted the copilot/fix-redis-authentication-error branch November 7, 2025 20:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ISSUE_251107_coaia_tash_fetch.md

2 participants